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WHAT IS CLAIMED IS: 



1 . A method for r\ overing 3D scene structure and camera motion from image data 
obtained from a multi-im^ge sequence, wherein a reference image of the sequence is 
taken by a camera at a reference perspective and one or more successive images of the 
sequence are taken at one or more successive different perspectives by translating and/or 
rotating the camera, the method comprising the steps of: 

(a) determining image dam shifts for each successive image with respect to the 
reference image; the shifts being (ferived from the camera translation and/or rotation from 

10 the reference perspective to the successive different perspectives; 

(b) constructing a shift data natrix that incorporates the image data shifts for each 

image; 

(c) calculating a rank-1 facte rization from the shift data matrix using SVD, with 
one of the rank-1 factors being a veltor corresponding to the 3D structure and the other 

15 rank- 1 factor being a vector corresponding to the size of the camera motions; 

(d) dividing the successive images into smoothing windows; 

(e) recovering the direction of camera motion from the first vector corresponding 
to the 3D structure by solving a limear equation; and 

(f) recovering the 3D stricture by solving a linear equation using the recovered 
20 camera motion. 



25 



2. The method of clainl 1 , wherein, step (e) includes: 
computing a first pnSiection matrix; 
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recovering c|mera rotation vectors from the shift data matrix, and the first 
projection matrix; 

computing a sec&nd projection matrix; and 

recovering the direction of camera translation using the shift data matrix, the 
reference image, the second projection matrix and the recovered camera rotation 
vectors. 



3. The method of claim 21 wherein step (f) includes recovering the 3D structure from 
the shift data matrix, the referAice image, the recovered camera rotation vectors and the 
recovered direction of translati )n vectors. 



The method of claim 1 
recovering the rotatioi 



further including preliminary steps of : 
of the camera between each successive image; and 
warping all images in |he sequence toward the reference image, while neglecting 
the translations. 



5. The method of claim ft, wherein step (b) comprises: 

computing H and A JL , where H is a (N p - 3)x N p matrix defined so that HH r is 
the identity matrix and H annihilates the three vectors x ¥ x9 ^¥ y , x ¥ z where the three 
vectors are computed from She reference image as 
^ = {vi • r (1) (p)}, ¥ v ^ {vl r (2) (p)}, ^ {V/ • r (3) (p) where 
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( -xy \ 



xy 



and . A is a shift data matim that gives the difference in intensities between each 
successive image and the reference image and is a (N f - l)x N p matrix with entries AI' n 
, where All is the change in (smoothed) intensity with respect to the reference image, 
5 and with no smoothing All=I - I° n , where N { is the number of images, N p is the 
number of pixels, and where /' denotes the /-th image, with i=0. 1 . . . , N f - 1 , and where 
f n = V [p n ) denotes the image Intensity at the «-th pixel position in/ 7 , where 1° is the 
reference image, where x and j are the image coordinates of the pixel position and 

p= (x,y) and where A CH = cF 2 AH T where C is a constant matrix with and where the 
10 notation {V} used to denote/a vector with elements given by the V a . 



U 

o 
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6. The method of claim 1, wherein step (c) comprises: 

computing a raijf-1 factorization of -A CH * M (1) S (l)r where M (,) ,S (,)r are 
vectors corresponding tp the motion and structure respectively. 



7. The methodpf claim 1, wherein step (c) comprises: 

computing/a rank-3 factorization of - A CH * £M (a) S (a)r where M (a) , S {a)T are 

I 0=1 

vectors corresponding to the motion and structure respectively; 

setting Z~ x as constant within each window, where Z is the depth from the 
20 camera to a 3D scene along the cameras optical axis; 
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listing each of the pixqg so that those in the k-th smoothing window have 
sequential indices nk, (nk+1), . . .\ n k+i - 0; 

computing a first projectiorfomatrix by computing a Np x Np projection matrix P n 
which is block diagonal with zero entries between different smoothing windows, and 
which annihilates the vectors {V/ • p}, ^ }and {/ v } where {V/} is a vector containing the 
gradient of the intensity at each pixel, a%i l x and \ y are the gradients of the intensity in the 
reference image in the x and y directions 

recovering the three camera rotation vectors includes solving the following 



equations 

P n (H r S w -W fl) )=0 for the 3- 
computing a second projection mkrix includes computing a N p x N p projection 



ector w^ ; 



matrix , which is block diagonal witW zero entries between different smoothing 
windows and annihilates (# r S (a) )-Wf (a) where w (a) is the vector recovered 
previously; 

recovering the directions of camera translations by solving for the directions of 
translation f ^ via 

P< a) (- fl a) {l x }--fl a) {l y }+ 7l w {p • ^})= 0 and; 
recovering Z n via 

[H T S {a) \ -[¥w {a) \ =r w (z; l J^ w p„ -[f M \)-VI n ) where [f^ represents f < a) f < fl) 
the x and y component of the/franslation direction, and where are constants. 



8. The method of claim 1, wherein step (d) comprises: 
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setting Z n 1 as constant within each window, where Z is the depth from the camera 
to a 3D scene along the cameras optical axis; 

listing each of the pixel! so that those in the A>th smoothing window have 
sequential indices nk, (nk+1), . .1 (nk+i - 1). 



10 



9. The method of claim 2, wherein the step of computing a first projection matrix 
includes computing a Np x Np p "ojection matrix P n which is block diagonal with zero 
entries between different smootl ing windows, and which annihilates the vectors 
{W • p} 5 {l x } and {l y } where {w j is a vector containing the gradient of the intensity at 
each pixel, and I* and 1^ are the gradients of the intensity in the reference image in the x 
and y directions. 
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1 0. The method of claim 29 wherein the step of recovering camera rotation vectors 
includes solving the following equation 

P n (h 7 S (1) - ¥w) = ojfor the 3-vector w. 



1 1 . The method of claim 9 wherein, the step of computing a second projection matrix 
includes computing ai^lx^ projection matrix P T 5 which is block diagonal with zero 

entries between different smoothing windows and annihilates - *Fw where w is 



20 the vector recovered nreviously 
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1 2. The method of clai^ 2 wherein, the step of recovering the direction of camera 
translation includes solving ipr the direction of translation f via 

p f (-t{^}-nkv}+7;{p-Y*) =0 - 

5 13. The method of claim 3|wherein, step (f) includes, recovering Z„ via 

(H T S {l) ) n -[¥w]„ =Z;itp n VfJJ-W,, where [f^ represents f x f y thexandy 
component of the translation direction. 
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